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ABSTRACT 

tt, a ^-4.- i This executive summary presents the major contusions 
of the initial feasibility and design phase of a two-phase stu'v to 
determine the outcome of referrals and placements and the actuai wage 
and 30b duration of selected employment service placements. The first 
chapter details the study scope and activities and describes the 
study design for each of the two phases. An exhibit summarizes the 

5 i? 9S *° J he phaSG 1 anal Ysis, presents a discussion of the 
? I findin 9 s on local office operations, and specifies the 
design features incorporated into the proposed demonstration 
projects. Chapters 2 and 3 present the methodologies to be 
demonstrated on a pilot-test basis at one local office for 
referral/verification follow-up and one local office for placement 
follow-up. Chapter 2 describes the objectives and scope of the 
proposed referral/verification demonstration program and the 
SiSfSiJ? operation concepts, presents procedural and organizational 
highlights, and identifies evaluation criteria that will be used to 
measure its effectiveness. Chapter 3 describes the placement 
follow-up demonstration, including its objectives and benefits 
operational highlights, methodological considerations, and criteria 
to be used in evaluating its performance. Benefits from the analyses 
of both demonstrations are listed in the fourth chapter on analysis 
activities. (YLB) 2 
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PRECJS 



The Office of Policy, Evaluation, and Research (OPER) and the United States 
Employment Service (USES), in conjunction with Macro Systems, Inc. (Macro), are 
currently investigating improved techniques to (L) determine the outcome of referrals 
and (2) determine the actual wage and job duration of selected Employment Service 
(ES) placements. This two-phased study includes collaboration with eight State 
Employment Security Agencies (SESAs) and knowledgeable Federal staff. 

The first phase, consisting of a feasibility study and development of a detailed 
demonstration project design, has been completed. This Executive Summary presents 
the findings and conclusions of the research team as well as an overview of the opera- 
tional design for both the referral/verification and placement follow-up demonstrations. 

The second phase will involve implementing the demonstration programs in two 
separate ES local offices during the third quarter of Fiscal Year 1930 and monitoring 
and analyzing their impact. Findings and conclusions will be drawn from these efforts 
and, further, recommendations concerning the follow-up processes will be formulated 
for national office review and incorporation into national-level policy, as appropriate. 

Once implemented and evaluated, it is anticipated that the demonstration pro- 
grams will yield significant benefits to the Federal/State ES service delivery network 
including: 

Referral/Verification Demonstration 

An increase in the number of referrals verified and placements captured 

More effective staff utilization through reductions in unsuccessful employer 
contacts and a delineation of employer-related and applicant-related respon- 
sibilities 

Better employer relations at the operational levels by utilizing staff knowl- 
edgeable about an employer's business and sensitive to the needs of the 
employer 

Placement Follow-Up Demonstration 

An additional source, on a sample basis, of local, State, regional, and/or 
national labor market information and USES performance data 

An improved ability of ES local offices to determine accurately and subse- 
quently to obtain credit for higher quality placements 

An aid to ES local offices to interpret and assess job order specifications 
and, therefore, result in better order servicing and fewer cancellations of 
openings 

The chapters that follow detail the overall study objectives, specifications of 
both the referrals and placements follow-up demonstration designs, and evaluation 
and analysis activities to be conducted for both demonstrations. 
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I. STUDY OBJECTIVES AND STUDY DESIGN 



The Office of Policy, Evaluation, and Research (OPER), in conjunction 
with the United States Employment Service (USES) of the Employment and Train- 
ing Administration (ETA), is currently investigating improved techniques for 
determining the outcome of referrals and placements and for determining actual 
wage and job duration of selected Employment Service (ES) placements. This 
Executive Summary presents the major conclusions of the initial feasibility and 
design phase and includes those methodologies to be demonstrated on a pilot- 
test basis at one local office for referral/verification follow-up and one local 
office for placement follow-up. 

1. STUDY SCOPE AND OBJECTIVES 

The study focuses on the development, demonstration, and evaluation of 
techniques designed (1) to provide ES local office staff with referral results 
information in a more timely and accurate manner and (2) to provide local, 
State, regional, and national program managers and policy formulators with data 
on actual wages and job duration for a sample of ES placements. More specifi- 
cally, the study is designed to accomplish the following objectives: 

To provide insight into current ES follow-up procedures and 
problems 

To develop and test potentially superior, ES referral and placement 
follow-up systems 

To analyze data on ES referrals and placements obtained through 
the conduct of demonstration programs 

To determine the actual wage and job duration of ES placements 
over time through the use of sampling techniques 

To formulate recommendations for ES referral and placement 
follow-up activities as well as to develop a methodology for a 
broader pilot or national-level field test 
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The data collected and analysis performed during the study encompassed all 
local office components impacting on the placement process. Excluded from the 
analysis were activities related to and in support of the WIN program as well as 
non-ES-related CETA activities. 

2. GENERAL STUDY DESIGN 

A two-phased approach for conducting the study has been designated as 
follows : 

Phase I— Feasibility Study And Demonstration Design 

During Phase I, eight case studies (i.e., ES local office reviews) were con- 
ducted in order to gain in-depth insight into current ES follow-up procedures 
and problems. Six major areas of local office operations were examined: 

Organization and staffing 
Applicant and service flows 
Employer services relations 

Operational and management information systems and report 
verification 

Referral/ verification procedures 

Problem areas, including coordinative, operational, and peculiar 
local office demographics affecting referral verification and place- 
ment follow-up systems 

This information was analyzed and alternative follow-up methods were identified 
and evaluated and subsequently used in designing the recommended referral/ 
placement follow-up systems that will be tested in the second phase of the proj- 
ect. Exhibit I, following this page, summarizes the major findings and conclu- 
sions of the Phase I analysis, presents a discussion of the impact of the findings 
on local office operations as well as specifies the design features incorporated 
into the proposed demonstration projects. It should be noted that post-placement 
follow-up systems, other than placement validation, were non-existent in the case 
study sites and, therefore, excluded from discussion in Exhibit I. 
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EXHIBIT 1(1) 



tmploynwnt uud Training Administration 
Referrals/ Placement Demonstration 



MB1UNSTMTION FINDINGS AND CCNCIUSIONS 
CLASSIFICATION CROSSWALK TO DESIGN NATURE 



Aitd o) Sped* I baium 



1, Organuatioii, Mauageineut, 
oimJ Mailing 



Finding anil Cuudusioni 

(1) Staff responsible for verifying referrals »I*ollU be laiiullar 
Willi built the uider taking and reletting junction* 



(2| Referral vesication approaches should be tailored to 
conform wllh local office euvhoumeuts 



(3) The assignment of employer account responsibility lo Inter- 
viewer* appeal* lo have j number oi advantage! jor veriiica- 
lion a* well ai ior providing setvlees to applicants and 
employer* 



(4) Rotation of vurificjiluu rc*i>oiisiblliiy among staff should be 
miulinited 



(5) Initial verification should be sdicduled within a matter of 
(iayi alter Hit referral in order to maximize services available 
lo applicants uA employes 



) Subsequent verification contact* should also be scheduled on j 
timely baits 



Impact on Durational Environment 



, Allow verillcaliou staff lo combine order taking and Veriil- 

caiion Into one employer contact 
, Permits verification slaf/ to asses* employer need* and 

applicant skills belter 



, Penults elficieul utilisation of staff by office management 
, Permit) efficient iniliutlou of available systems and reports 
Capabilities 



, Enhances the office's employer relations efforts 

, Allows office staff to Identity employer needs tetter 

. Tamil* verification stalf to combine verification and order 

taking into ouc employer contact, dicieby mmlmblng 

employer overcount 

. Negative impact on employer relations with rotation 
. Difficult for management to gauge the success/ failure of 
verification procedures with rotation, i.e., J liferent 
personality types Involved, difficult to track down res|>oiislblc 
parlies when problems occur, etc. 

, tawes tltat Job orders artscrviccd on a timely basis 
. Increases the integrity of verification data obtained 
, Conveys the Impression that job Service is genuinely con- 
cerned and receptive 10 the employer's needs 
(professionalism) 

, EiLiues that Job orders are serviced on a timely basis 
, Increases the integrity of verification data obljiucd 
, Prevents verification staff from having 10 "catch up" on 

numerous referrals remaining unverified for extended 

periods of lime 



Demonstration Dcsbi Considerations 



'Jltc design includes an employer account responsibility 
feature whereby Employer Account Specialists perform 
order taking and verification fuuctloiti simultaneously, 
'ihese staff have iulcrylcwer experience and are familiar 
with referral and cider taking functions, 

Of necessity, (he demonstration uielJtodology must have 
jiotctKial applicability lo a wide variety ol offices; however, 
It is envisioned that minor changes can be Incorporated in 
the design for more efficient operations in any specific 
office. 

'Die design emphasises employer service account responsi- 
bility. Employer overeoutaci and duplication of eifori 
arc uduimized. 



Verification functions art performed by Employer Accoiud 
Specialist, No roution is envisioned hi the 
demoust ration design. 



The initial employer contact Is performed within 24 hours 
of Ihc referral, unless otherwise specified by llie employer* 



The design features a definitive lime schedule for Coir 
dueling verification contacts. 
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EXHIBIT 1(2} 



2. huici luiial and Service Delivery 
Components 



rbuliugaud Cuiuliuiujis 



(1 ) Employers j|>|>car lo be both the most common and the most 
reliable sou ice for verification J -id 



(2) Telephone verification iippcau to be llie mott efficient 
Method of oUainiug referral feedback Irom employers 



(3) Additional or secondary source* tor obi alniug referral results 
should be milked to maximlje data iplity where cost 
effective 



(4) Educating employer* ill tiic objective* and procedures for 
verifying referral JiOuld be Incoqwaied hi available 
employer service programs: 



(S) Addulonal employer services, such J* Ouier servicing, order 
t .iking, employer reliilioiis, etc., should be conducted with 
vesication to minimize employer contacts 



(6) 'W t referral verification effort sliould nut decrease ik ability 
of -l jit to service wallwn tulllc and laLe job orders 



Impact on Operational Eiwlruumcul 

, Minimizes the reporting of invalid data to the automated 
system 

, Reduces the number of additional contacts during the verifi- 
cation process, i.e. , reverifying applicant data with 
employers, contacting employes when applicants are onsuie 
of the Interview results, etc. 

, Permits timely verification with respect to the referral 
, Cost elective with respect to stall utilisation 
, Allows verification stall to determine tk availability status 
of verification Contacts imiautiincotisly 

, Ensures dial all available methods and souicti of contact are 
utilized 

, Allows office staff to identify employers rci|uiiing ESR 
contacts 

, Eidunees employer relations 

, Increases the verificjliou success rale 

, Cost effective with relation to staff utilisation, I.e., easily 

assimilated Into employer present at lorn made by ESRs 

during site visits and/or telephone coutacU 

, Cost effective with relatiou to staff utilization 
Eidiances employer relations, I.e., mini mizes employer 
overvunlact 

Ensures that onJcis ait serviced lu a timely fashion 

Implies that verification duty assignments be the responsi- 
bility of staff not Involved with applicant registration, ft ferral 
Ensures that applicants are properly registered aid aflonled all 
available Services 

Allows referral staff to become more personable with appli- 
cants, 1. e. , Improves applicant's view of |ob Service as well 
as Increasei motivation 



Demoiiilration Design Considerations 



Employers are the primary source for obtaining referral 
i. rails, 



Employers are contacted by telephone by the Employer 
Account Specialist unlets otherwise specified. Employ- 
er* do have the option to choose their preferred method of 
cowact, I, c. , return of ES-SOB card, ESR site visit, etc. 

The design features alternative procedures forobtaiuiug 
referral results when (he Initial contact cannot be nude or 
ll unsuccessful , 



'llie design emphashes employer urlenlatlou In follow-up 
procedures conducted by jot Service, tluou^ the job 
Service Improvement Program (|SIP) and/or other 
employer gwups. 



Tk design integrates verification activities widi these 
oilier employer service functions, 



Ik design separates staff responsible for verification and 
account servicing from placement interviewer staff. 
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EXHIBIT 1(3) 



2. functional M Service Delivery 
Cumponeuts (loutmued) 



3. Operational and Management 



Huding and Cuidmlaus 



(7) lleferral verification should include obtaining wage dala on 
hires jiJ stulc reasons for upplli: jjiU not Lcii^ hired 



(8) Interviewers should systematically be uulilled o! qualitative 
information obtained from employers 



(1) Maximum me should be iujJc uf jvjUuUc technology and 
resource* 



(2) A jyUciujiicaHy ^uUuceJ ijucuiug device sluulJ be avail- 
able daily to identify unverified lefcrrals requiring 
verification 



(3) A dally procedure/device la notify interviewer staff of 
referral residlj ihould be available 



(4) Timeliness and accuracy in rtcoidlng and entering veriilca- 
lU data to lie automated system are of ibc utniotL 
hii|MiaiiCe 



Impact un Operational Envlrouneol 

, Increases ability ol referral staff u> asseH employer needs 
. AwljiU management staff lu identifying interviewers requiring 

assistance, i.e. , those who refer applicants not (jualilicd fur 

the job opeidug 

. Increase! ability of referral staff to -saetf employer need* 
. Agists management stall in idcitlilyiiig liilcrviewers requiring 

ossblauce, i.e., those who refer applicants not qualified fur 

iJic job opening 
, Enhances eiupluyer relations, i.e. , Improves employer* 1 

perce|4ion oi job Service 

, Cost effective 

, Increases tlie probability that referral inforinatiou will be 

repuricd lu tlie automated system in J timely maimer 
, Permits ellicleiit utilization uf staff 

, billet that the liiitijl verification Mc\u\A is performed In a 

timely maimer Willi resect to t)ie refusal 
, Reduces employer OvercoiiUctj i.e., einpluycrs who have 

already verified outstanding referrali using another method 

of contact are not recontaclcd 

, Morales interviewer siaff to relcr and/or provide applicants 
with positive service 

, Reduces employer overcoutaetj i.e. , employeis who have 
previously verified referrals using normal chaiuicb of couimu 
nicatiun arc not recuutacted wlieii micuing device liavallablc 

Ensures that rele k lng interviewers art informed of referral 
results lu a timely fashion w{Ji reject to die referral 
Reduces employer overcoutaclj I.e. , employers who have 
verified in accordance with Job Service expectations f me not 
reconl acted by verification stall 11 recording is timely 



Demoustraiiun Design Comjldemtluns 



Ik design specifics that Employer Account Socialists 
routinely attempt to obtain this information fioui employer* 



Employer Accoimt Specially notify referring Inter/ JWiin 
Of this information using a redesigned tS-SOa, Employer 
hurodnctory Card, 



Tim design incurporaick available ollice resources, i 
data entry equipment, source documents, etc. 



Referral records are maintained to permit staff the con- 
tinuous appraisal of die status of unverified referrals. 



Referral remits are obtained by inferring ijiicr* 

viewers using lite ES-SG3 and the job Bank Dally Applicant 

Transaction Keport ( 



lleferral results ait posted Immediately lo liijiii docuuieitts 
and ore forwarded daily lor dalu entiy, 
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Phase II — Implementation And Evaluation 



Phase II will involve implementing the recommended T^llow-up systems in a 
selected site(s) and monitoring' and analyzing their impact. Findings and con- 
clusions will be drawn from these demonstration projects, and further recom- 
mendations concerning the follow-up processes will be formulated and incorpo- 
rated into national level policy, as appropriate. 

To satisfy the general objective of the study, i.e., design and pilot test 
preferred follow-up strategies in environments that are typical of the ES uni- 
verse, the following research parameters have been declared: 



The study is a pilot test with no inferences about the universe 
of ES offices and environments. 

The primary demonstration methodological concerns include cost- 
effectiveness and operational performance. 

Data collected by the demonstration systems will be analyzed; 
however, no particular behavioral models, such as impact on 
special applicant populations, applicant attitude, etc., will be 
investigated. 

The demonstration time period for observation and evaluation is 
Limited. 



Analyses to be performed on data collected during the referral and placement 
demonstration include: (1) cost-effectiveness, (2) referral process dynamics, 
and (3) wage and job duration experience of placements. 
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II. REFERRALS/VERIFICATION DEMONSTRATION DESIGN 



This chapter describes the objectives and scope of the proposed referral/ 
verification demonstration program and the underlying operational concepts, 
presents procedural and organizational highlights, and identifies evaluation 
criteria that will be used to measure its effectiveness. 

1. OBJECTIVES AND SCOPE 

The primary focus of the referrals /verification demonstration project is to 
test and evaluate improved methods of obtaining referral feedback information in 
a timely and accurate manner using a method of contact that is mutually agree- 
able with the local office and the employer. The proposed demonstration is 
designed to accomplish the following objectives: 

To demonstrate improved methods of obtaining referral feedback 
information 

To gather referral feedback information in an expedient and cost- 
effective manner 

To improve the validity of information obtained 

To establish clear lines of responsibility for all components of 
the verification process 

To establish improved relations with both applicant and employer 
communities through better and more professional levels of service 

The demonstration encompasses all ES referrals exclusive of those WIN and non- 
ES-related CETA referral activities. 

2. OPERATIONAL CONCEPTS 

During the review of the referral/placement process in the case study 
sites, it became clear that more service-level integration, accountability, and 



ERLC 



-7- 

14 



structure were required to satisfy the needs of both applicants and employers. 
These fundamental Concepts have been incorporated into the proposed demon- 
stration and are presented in the descriptive summaries that follow: 



Integration Of Employer Services — ES local office staff performing" 
order taking, verification, placement interviewing, and job devel- 
opment and ESRs all routinely contact employers — in some cases, 
they contact the same employer more than once per day. The 
proposed design encourages local office staff performing employer- 
oriented functions, with the exception of job development and 0 
placement interviewer staff, to conduct all employer contact 
through an Employer Account Specialist (EAS). 

The EAS contacts employers for purposes of referral verification, 
order taking, and job- order servicing. These telephone contacts 
are scheduled at a mutually agreeable time with the employers. 
A maximum of one employer contact daily is necessary to service 
existing job orders, record new job orders, and obtain referral 
follow-up information. 

In addition, an ESR formally assists the EAS in these areas and 
continues to be responsible for other non-operational employer- 
oriented activities such as coordinating JSIP activities, marketing 
ES programs, and assisting employers in technical areas. 

Operational Accountability — In addition to the employer service 
integration concept, additional levels of ES staff accountability 
for employer account servicing have been incorporated into the 
design. Organizationally, employer service teams, each with 
responsibility for servicing a selected group of employers and 
comprising EASs who have individual employer account responsi- 
bility, perform the fundamental account servicing functions. As 
such, and in conjunction with an ESR who is" also assigned to 
the team, EASs provide employers with a single contact point 
within the ES local office to which all operational communications 
flow. Order taking, revisions to job openings, verification and 
follow-up, and other day-to-day operational functions flow to and 
from members of each team of Employer Account Specialists. 
Using this approach, greater control over employer contacts, 
including referral control; better control of job orders and open- 
ings; and, presumably, more effective account servicing can be 
provided to the employer. 

Structure And Time Frames — One major question confronting a 
verification technician is if and when the employer will return 
the referral results. The answer to this question is unknown, 
and, because it is unknown, staff initiate telephone contacts to 
solicit the results. Often, however, these telephone attempts 
are made and produce no results, yet staff time has been 
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expended. In the proposed design, attempts are made to test 
the feasibility of scheduling employer responses both in terms of 
time and method (telephone or mail). It is hypothesized that, if 
the employer establishes the method and time frame, he/she will 
be more likely to respond in a timely manner. 

In addition to the employer's preferred method for feedback, the 
design incorporates supplementary methods that include the use 
of SSRs and applicants as potential referral feedback contacts. 
Each of these methods is used under specific circumstances and 
time frames. 

Last, it is implied in the proposed design that, through implementation of these 
concepts and the other process-related techniques proposed below, local office 
staff in general will become more sensitive to the needs of employers, and the 
employer services unit staff in particular will become more knowledgeable about 
the employers 1 businesses. Further, with tighter levels of operational control 
on referral and follow-up activity, employers will perceive a greater level of 
staff professionalism and so will the applicants. 

Exhibit II, following this page, presents a procedural flow diagram of the 
demonstration design. 

3. HIGHLIGHTS OF THE PROPOSED DEMONSTRATION 

This section presents the key operational components of the design. 

(1) Revised Employer Introduction Card/Reporting Document (ES-508) 

The proposed design uses a revised three part Employer Introductory 
Card/Reporting Document (ES-508). This card is designed to: (1) mini- 
mize document handling in the local office, (2) provide information neces- 
sary to identify unverified referrals, (3) provide referral feedback to 
placement interviewer staff, and (4) report referrals and referral feedback 
to the automated system. In addition, each part of the ES-508 card con- 
tains text intended to familiarize the user with its purpose. 



EXHIBIT ll(ty 

Employment and Training a A-m +n Jyn-r» rfTi 
Referral*/ Placement Demonsnatiou 

PROCESS FLOW DIAGRAM 



APPLICANT SERVICES UNIT 

Match Applicants 
and Openings 
. File Search 
. Job Development 



j 

' Review Suitability of 

Job Opening with 
Applicant 

1 

interviewer Obtains 
• PennJsxion to Refer from 
I Referral Control 



Referring Interviewer 
Establishes Appointment 
with Employer 



Referring Interviewer 
Makes Referral 



Applicant introduction/ j 
Return Card (Top Copy, 
ES-S08T) Given to Appli- I 
cant to ?resent to I 
Employer j 



Referring interviewer 
r r epares Three Part 

Employer Introduction 
Card/Reporting 
Document (ES- 508) 



t Applicant Statistical Input 
I Card (Middle Copy, 
"j ES-S08M) Used to 
. Record Referral in System 



I 



{ Card Used to Report 
{ Referral Results When 
| Rammed by Employer 



Applicant Referral File j 
| Card (Bottom Copy, ( 
j ES-508B) Forwarded to | 
Account Responsible Team I 
for Identification of 1 
Outstanding Referrals 



Input/ Forward 

Card to 
Data Entry 



j Reports 
ing 



rVerifv-} 
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EXHIBIT 11(2) 



EMPLOYER SERVICES UNIT 



Vanfy-_ 




IDENTIFYING REFERRALS J 



TO BE VERIFIED 



Referral Results 
from Employer! 
-Vho Prefer Mail J— ' 
'/■rifi carton 



Employer Account ' 
Specialist Ida art- ! 
tie* Referrals - 
Reqairm* ' 
'/■rlflcirtoa 

i 



Ramm ES- SOSTj 
to Reia trine; 
'.ntervtewar 



Is Referrals 
Verified? i 



FILE RECORD 

KEEPING 
DOCUMENTS 



T 



E5- 51:* 



Active Applicant 



TTT7 



inactive 
* pplicaat 
File 



ZS-SUi 



Span Oraar 
TUa 



POSTING AND 
RECORD 
KEEPING 



L 



Prepare Racord i 
Keeptat; Document- ,*" 
ration from ES- 50SBI 
Card 

15-511 , £5-514 - 



Refetral Result 
Posted co ES-511 
by Office 
3acepcioni*t 



Referral Result 
Posted » ES-514 
by Clerical Staff 
in Employe* 

Services Unit 



| INPUT DATA I 
I ENTRY ! 



: Referrals Verified . 
by Pro fa trad .Method! 
of Employer Contact} 
the Day of Referral I ' 



PERFORM FOLLOW-UP I 
AND OBTAIN RESULTS I 



*1 



Referral Results 
Determined T 



Yea i So 



Haa Applicant 
I Reported for Work?! 



A Maximum of Tvrai 
Additional Coma eta t_ 
j Cjia? Employer's 
M Preferred Method 

Attempted Within . 

Four Days Foloww 
I ag Referral i 



Refexral Raul a 
Determined? 



j ESR Determines I 
j Alternative Methodj 
jof Employer Contact 
' and Perform Con- j 
Iocs Within 12 DarH 
(Following Referral I 



[Mail Contact 



Confirm with 
Employer After 
or on Scare Data 



Referral Results 
Noted cm ES-3088 



I Teleonona Contact I 
ESR Sice Visit 1 , 



Referral Results 
Detarratnad? 



I EAS ! 

: Attempci to Obtain | 

Referral Raauit — 
i trom Applicant 
■ Within 19 Days j 
! Following Referral j 
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Returned to 
• Employer Account 
Specialise 



-j 



'Applicant Camnc r Telephone 
Applicant Contact Mall 
Secondary Source ■ 
of Referral — " 

Contact *~* 



fRatenel Results) 




t 
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?.ai eiral Result 
Determined? 



ES- :OSB Renuned 
to EAS to Geek 
Data Entry 



ES- 508B L'sad co 
Enter Referral 
Results 



'.nput/ Forward 
Referrals co 
Data Entry 



ES- 508 M Used co 
Enter and Check/ 
Correct Data 
Entry 



ES-508B Card 
s lled in DUporit*oni_ 
In Cuestion File 



Six Ntocth * N - 
\^ Purye oi File j 



'A 



£5-5083 Forwarded \ 
\ to Referring, 
.ntepriewer 



t 



IS- 308 B FUed In 
Koldma Fue 



Probable Placement! 
Listing Ciecka 
Periodically 
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The three-part ES-508 card is described below: 



Applicant Introduction/Return Card (ES-508T) --This card has 
four basic functions: (1) to identify the applicant as a referral 
from Job Service, (2) to provide the employer with explanatory 
text pertaining to the verification process, (3) to emphasize the 
importance of obtaining referral feedback information, and (4) to 
provide the employer with a vehicle for notifying Job Service as 
to the status of the referral. This card is prepared by the 
referring interviewer at the time of referral and given to the 
applicant to take to the employer. 

Applicant Statistical Input Card (ES-508M) -- This part of the 
card is used to enter the referral into the automated system. 
Referring interviewers forward this card to data entry at the 
time of referral. The day following data entry, data entry staff 
verify the data input using this card as the source document. 

Applicant Referral File Card (ES-508B) — This part of the card is 
designed to serve two basic purposes. The Applicant Referral 
File Card is used by verification staff as a queuing device to 
identify unverified referrals. This card is completed by the 
referring interviewer at the time of referral and forwarded to 
verification staff. The reverse side of the Applicant Referral 
File Card provides verification staff with a record-keeping 
instrument to document verification events. This side is entitled 
the Verification Record. Verification attempts and methods of 
contact are recorded on this side in chronological sequence. 



Although it is not envisioned or intended that the redesigned ES-508 card 
will improve the employer, verification response rate, it is expected to have 
a positive effect on employer relations. 



(2) Employer Account Teams Are Responsible For Servicing And 
Coordinating All Employer-Oriented Activities 



Employer account responsible teams comprising Employer Service 
Representatives (ESRs) and verification staff perform all employer-oriented 
activities. The responsibilities of verification staff on this team include: 

Obtaining information on Job Service referrals 
Servicing job orders for assigned accounts 
Taking orders 

Performing employer relations services 
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ESRs assist verification staff in obtaining referral feedback when referral 
feedback cannot be obtained using the employer's preferred method of con- 
tact. In addition, ESRs principle functions remain: 



Soliciting new employer accounts 

Coordinating employer relations programs 

Providing technical assistance to employers with problems 

(3) Preferred Methods Of Employer Contact Arz Established 

The design allows employers to choose either mail or telephone as the 
preferred method of contact. The method of contact is determined at the 
time an order is taken for new accounts and confirmed at the time an order 
is taken on existing accounts. 

(4) Specific Time Frames For Obtaining Referral Results Are Established 

^. 

The initialN^erification attempt, unless otherwise specified, is initiated 
within 24 hours of the a «£efeprai, using the employer's preferred method of 
contact. Additional contacts are scheduled using these guidelines: 

One To Four Days Following Referral —Verification staff attempt 
to obtain referral feedback using the employer's preferred method 
of contact. 

Five To Twelve Days Following Referral — ESRs attempt to obtain 
referral feedback either using the employer's preferred method 
of contact or an alternative method of contact selected from avail- 
able employer information listings. 

Thirteen To Nineteen Days Following Re ferral— Verification staff 
attempt to solicit referral feedback from" applicant either by mail 
or telephone contact. 

Last, probable placement listings are used when available to identify 
outstanding unverified referrals. 
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4 - ORGANIZATION AND STAFFING OF PROPOSED DEMONSTRATION DESIGN 
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The proposed design identifies and organizes employer-oriented and 
applicant-oriented functions into two separate units. This functional segrega- 
tion facilitates the monitoring and evaluation aspects of the demonstration 
project and provides for a simple and controllable flow of information throughout 
the office. Exhibit III, following this page, is an organizational and staffing 
chart that illustrates the functional differentiation and manpower components 
required to operate and evaluate this project effectively. 

5. DEMONSTRATION METHODOLOGY 

Pre-established site selection criteria have been developed to ensure the 
selection of an ES office representative of most local operating environments. 
Prior to final site selection, a predemonstration site review will be conducted to 
identify the current local office practices impacting on the referral verification 
system. In addition, this review will highlight potential problems in implement- 
ing the demonstration and allow State agency/local office staff an opportunity to 
determine from their perspectives the expected impact of the demonstration on 
their operation. 

As proposed, the demonstration project test will be implemented at a single 
site and operated concurrent with the existing (control) referral follow-up sys- 
tem. Employer account responsibility will be distributed alphabetically to verifi- 
cation staff in the test and control units. Verification staff in the control unit 
will obtain referral feedback information using the office's current referral 
follow-up system. Staff assigned to the test unit will obtain referral feedback 
information using procedures outlined in the demonstration workplan. This 
alphabetic assignment will result in an equalization of workload for both study 
units. Management orientation sessions designed to ensure compatibility of the 
demonstration with local office operations and staff training will be conducted 
throughout the demonstration period. 

Exhibit IV, following Exhibit III, provides an overview of the referral veri- 
fication methods proposed in this demonstration. 
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Employment and Training Administration 
ReitmJj/ Placement Demonstration 



APPLICANT SERVICES 



. Applicant Outreach 
. Reception/ Intake 
. Application Taking 
. .Applicant Interviewing 
. Counseling/ Testing 
. Job Development 
. ReJerral 
. Special Applicant 

Services 
. File Search/ Call lm 
. Job Information 

Service 




REFERRAL DEMONSTRATION MODEL 
LOCAL OFFICE STAFFING/ FUNCTIONAL 
ORGANIZATIONAL CHART 



! Employment I 
i Counselor 
i Staff j 



Veterans 

Staff 



! Special 
"i Applicants 
! Staff 



Receptionist 
Staff 



Clerical 
Staff 



Employer Account 

Specialist: 
Order Servicing 
Order Taking 
Verification 
New Accounts 
Referral Control 




Clerical 
Staff 



i Employer Acco un t 
{Specialist: 
. Order Servicing 
. Order Taking 
. Verification 
. New Accounts 
. Referral Control 



ESR(i) 
Employer 
Relations 
Verification 



Clerical 
Staff 



EMPLOYER SERVICES 
. Job Order Taking 
. Order Service 
Responsibility 
. Verification 

- Referrals 

- Work Start Dates 

. Job Service Improve- 
ment Program . 
. Employer Relations 

PlOgl'JTTTt 

. Referral Control 



* Number of and size of employer account r es pon s ible teams dependent on office rue. 



-15- 



ERIC 



22 



i 



EXHC!TiV(l) 

toiployuient and Training Administration 
Referrals/Placement Demonstration 

ES ItiSFEKRALS FOLLOW-UP HEl'IlODOIflGY— SPECIFICATION 



AIUIC'AHT SEKVICES UNIT 


Component 


Function 


Procedural Comments 


Ket'erring Applicants to 


(1) 


Hatch applicants and openings 


• Referring interviewers utilize all available resources in 


Available Job Opening 




, File Search 
. Job Develu£Mitent 


an effort to generate referral activity 

- Review newly received job openings 

- Review open job order fileMcroftche 

- Review JSMS III ti Applicant Matching Profiles 
■ Use JSMS IV A Applicant Search Inquiry 

. Referring interviewers contact employers to determine 
if employment opportunities exist for job-ready 
applicants when listed job openings are unavailable 




(2) 


Review Lhu suitability of the selected job opening 
and/or job development contact with the applicant 


. Referring inuiviewurs discuss the following job ypedflca 
this with the applicant prior to referral; 

- Skills required in job opening 

- Pay rate 

- Location 

- Unique duties assigned by the employer to this job 
class! fication 

- Physical demands 




13) 


Obtain permission to refer 


. Referring interviewer obtains permission to refer from the 
Employer Account Specialist (EAS). 




M) 


Establish appointment with employer 


, After obtaining permission to refer from the Employer 
Sun/ices Unit, the referring interviewer telephones the 
employer to schedule an interview appointment, This func- 
tion is precluded only when specified by the employer; in 
which case the applicant is sent without an appointment. 




(5) 


Refer applicant to suitable job opening 


, Referring interviewer advises the applicant of employer's 
address, whom to see, and the time of interview appoint- 
ment, 




16) 


Prepare three-part Employer Introduction Card/ 
Keportiny Document (ES-508) at the time of referral 
, Applicant IntroductWfleturn Card (ES-50BT) 


. Referring interviewer gives the Applicant Introduction/ 
Return Card to the applicant to carry to the employer, 
The Applicant Introduction/Return Card identifies the 
applicant as being referred from Job Service. 

. The Applicant Introduction/Return Card permits employers 
to verify referral results by mail, if desired, The 
reference text on this card is intended to solicit com- 
ments pertaining to the employer's decision to hire or not 
hire an applicant. 
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EXHIBIT IV(2) 



APPLICANT SERVICES UN IV \ 


Component 


Function 


Procedural Comments 


Heiferring Applicants to 
Available Job Opening 
(continued) 


(6) . Applicant Statistical Input Card (ES-500M) 

i 

. Applicant ket'erral Me Card (ES-MBB) 
. Verification Record (l£S-508b (reverse side!) 


. Referring interviewers forward this card to data entry to 
expedite the input of referrals to the automated system. 

. Data entry staff use this card to verify statistical data 
entered to the automated system the day following input. 

. Referring interviewers route this copy of the E5-508 card 
to the responsible Employer Sen/ices Unit Employer 
Account Specialist, 

• Eaployer Account Specialists m cards to queue the 
verification process. 

. Employer Account Specialists record verification 

attempts in chronological sequence, 




EMP1AYER SERVICES UNIT 


Verifying the Results of 
Referrals 


(1) Identify unverified referrals requiring verifica- 
tion 

(2) EASo attempt verification the day of referral 

(3) ESRs assist EASs with verification 


. Employer Account Specialists review the fiuployer 
Account Index to determine the preferred method of con- 
tact for all Applicant Referral File Cards received from 
the Applicant services Unit. 

. Employer Account Specialists review Application Intro- 
duction/Return Cards received by mail from employers and 
record referral results on their appropriate Applicant 
Referral file Card. 

. Contact is made using the employer's preferred method. 

. Unsuccessful initial verification attempts are followed 
up twice, within the next four days, using the employer's 
preferred method of contact, 

. In accordance with the Employment Service Manual proce- 
dures, EASs determine referral results anu report to work 
status. 

. EASs unite the Applicant Referral Pile Card to the ESR 
assigned to their unit when unsuccessful in obtaining 
referral results using the employer's preferred method of 
verification. 

. Within Lhe next seven days, the ESR determines the appro-* 
pt'iale alternative met lux) or* employer contact and per- 
furms contact. 
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EXHIBIT 1V(3) 





mimw SKHVicfcis unit 


Component 


Fund ion 


Procedural Comments 


Verifying the Results of 
Referrals (continued) 


(i) (continued) 


. ESRs record contacts and results on the Verification 
Record prior to returning the Applicant Referral Card 
to the ms. 

. Applicant Referral File Cards that remain unverified are 
returned to the appropriate EAS, 




W Verification attempted with the applicant 
(secondary source of verification) 


♦ EASs determine the method of applicant contact, i.e., mail 
or telephone, 

. Applicants are contacted within seven days after receipt 
of the Applicant Referral File Cardjrom ESRs. 




(5) Report referral results to Lhe automated system or 
file unverified referrals 


. For referral results that are obtained during tlie proce- 
dures detailed in functions 2, 3, and 4, the party obtain- 
ing tlie result is responsible for updating the Applicant 
Referral File Card and posting to record-keeping docu- 
ments and fowurding to data entry. 

. Referrals that remain unverified after applicant contact 
are filed by Lhe EAS in a Disposition in Question file 
maintained for Lhe unit vhU fiw> \a n\uu*\a*A i^ri^Ai,* 
ally against the Probable Placement listing, where avail- 
able, 




16) Post referral results to record-keeping documents 


. The receptionist wists the referral i>rhH tn fq-hi 
Application Card using the Applicant Referral Card, 

. Designated clerical staff, consistent with the office's 
organisational structure, posts tlie referral result to 
the ES-514 Job Order using the Applicant Referral Card. 
This is performed immediately after the receptionist 
posts referral results to the ES-511. 




17) File record-keeping document a 


. uipun. Lca^iuific lui ^aLiiiy rererrai results to Lecoru- 
keeping documents are also responsible for refiling 
source documents. 




18) Input reterral results to the automated system 


. Applicant Referral File Cards are forwarded to data entry 
immediately after referral results are posted to the 
record-keeping documents. After entry, these cards are 
leturned to the responsible HAS. 

. EASs verify entry of the referral result to the automated 
system the following day. 
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EXHIBIT IV(4j 



mnma amicus unit 


Coiu£Xifient 


Fund ion 


Ptocedura* Confluents 


Verifying the Results of 
Referrals (continued) 


(9) Notify the referring interviewer of information 
pertaining to the disposition of referrals 


, The m forwards the Applicant keferral FUu Caid to the 
referring intuVvjewcr for review, 

. In locations with Job Bank, the daily Applicant Trans- 
action Listing can be used, in addition to the Applicant 
Referral file Card, by referring interviewers to obtain 
referral results. 
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6. EVALUATION CRITERIA 



Evaluation factors to be used during the demonstration are discussed in 
this section. These factors will be utilized in gathering sufficient data to evalu- 
ate the impact of the demonstration on local office activities. The primary areas 
of evaluation include: 

Effectiveness Evaluation —The objective of this evaluation is to 
measure the impact of the dynamics of the demonstration on local 
office operations. The primary areas of effectiveness evaluation 
include: 

Reduction in time to determine the results of referrals 

Reduction in employer contacts required to obtain referral 
feedback 

Improvement in percentage of referrals verified 

Programmatic Evaluation —The objective of this evaluation is to 
assess the performance of both the control and test groups. As 
such, the following performance indicators will be used: 

Proportion of active applicants referred 

Proportion of openings filled 

Proportion of total referrals verified 

Proportion of total referrals unverified after 19 days 

Proportion of total referrals verified, by method (telephone, 
mail, site visit) 

Cost-Effectiveness Evaluation — The objective of this evaluation is 
to determine whether the test group is more cost-effective than 
the control group with respect to the following: 

Verification staff hours per attempted verification 
Verification staff hours per successful verification 
Total staff hours per attempted verification 
Total staff hours per successful verification 

Organizational Analysis — ES staff and employer attitudes regard- 
• ing the test referral verification demonstration system will be 

obtained through personal interviews and JSIP committee meetings. 

* * * * 
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HI. PLACEMENT FOLLOW-UP DEMONSTRATION DESIGN 



This chapter describes the placement demonstration, including its objectives 
and benefits, operational highlights, methodological considerations, and criteria 
to be used in evaluating its performance. The methodology, as proposed, is to 
be tested in a single ES local office. 

1. OBJECTIVES AND SCOPE 

The primary focus of the demonstration project is to test a proposed meth- 
odology for following up on the wage and job duration experience of sample ES 
placements. The two principal objectives of this methodology are to: 

Obtain data on actual wages and job duration of sample ES 
placements 

Compare actual wages and job duration with those originally 
specified in job orders 

In addition to these objectives, several benefits may be derived from this 
methodology. For example, after testing and refinement, the methodology may: 

Improve the ability of ES local offices to determine accurately 
and subsequently to obtain credit for higher quality placements 

Assist ES local offices to interpret and assess job order specifi- 
cations and, therefore, result in better order servicing and fewer 
cancellations of openings 

Provide, on a sample basis, an additional source of local, State, 
regional, and/or national labor market information and USES per- 
formance data 
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2. OPERATIONAL CONCEPTS AND HIGHLIGHTS 



Exhibit V, following this page, displays the major operational concepts 
underlying the methodology, and Exhibit VI, following Exhibit V, summarizes 
the highlights of the demonstration. The proposed placement follow-up method- 
ology is a two-stage process designed to (1) minimize local office impact, (2) be 
independent of the local office environment, and (3) be flexible in scope and 
application. Validation by ETA standards of the sample placements is incorpo- 
rated into the methodology, and the sample size and follow-up procedures ensure 
statistical significance and reliability. Exhibit VII, following Exhibit VI, pre- 
sents the process flow of the demonstration. 



Several methodological considerations have been addressed during the 
development of the the placement follow-up demonstration design. The three 
most important are presented in the summary descriptions below: 



Site Selection Criteria —Specifically, the selected site must be a 
full service office, year-round in operation and diversified to 
provide a representative sample of ES placements, and must use 
and support ADS and/or Job Bank Systems. 

Training And Technical Assistance — On-site training both prior 
to implementation and during the operation of the demonstration 
follow-up system will be conducted by project staff. Training 
will be performed in conjunction with appropriate ES staff. 

Project Team Staffing — Identification of staff to perform demon- 
stration activities will be accomplished in a manner to ensure 
minimal disruption to local office operations. Project team mem- 
bers will be responsible for performing all demonstration activi- 
ties with the exception of: (1) providing necessary documents 
for placement sample selection, (2) performing follow-up calls to 
nonresponding employers, and (3) completing selected measure- 
ment and control forms. 



Exhibit VIII, following Exhibit VII, provides a detailed specification show- 
ing the function and subfunctions , related documents, and procedural details 
associated with the placement follow-up methodology. 
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EXHIBIT V 



Employment and Training Administration 
Referrals/Placement Demonstration 



PLACEMENT FOLLOW-UP METHODOLOGY — 
OPERATIONAL CONCEPTS 



Two-Stage Follow-Up Process —Wage , job duration, and placement 
validity data are collected at two points in time, at intervals 
consistent with current ESARS reporting conventions, to obtain 
both short- and long-term information on a sample of ES placements 
at a local office. 

Minimum Local Office Impact — The methodology is designed to minimize 
the workload required of local office staff, the disruption of local 
office operations, and any adverse effect on applicant or employer 
services, and relations. In addition, both the local office selected 
and the local area employers are given advance notification and 
instructions. 

Independence From Local Office Environment — The placement follow-up 
methodology can operate under any organizational structure without 
the presence of automated capabilities and without either ADS or 
Job Bank being available. 

Flexibility In Scope And Application — The methodology can be operated 
at any level of magnitude, including the specification of larger or 
smaller sample sizes, the analysis of desired subsets or programs, 
and/or the simultaneous operation of the methodology at more than one 
office. In addition, the methodology can, if desired, be linked with 
either the referral follow-up methodology proposed in this report or 
the current ETA local office validation surveys. 

Statistical Significance And Reliability —The proposed sample size is 
more than twice that normally required by ETA for local office surveys, 
nonrespondents are prompted by telephone, and short-term follow-up 
data are validated during the long-term follow-up. 

Built-in Sample Placement Validation —As part of the follow-up process, 
sufficient data are collected to permit the determination of the validity, 
by ETA definition, of the sample placements. 




EXHIBIT VI 

Employment and Training Administration 
Referrals/Placement Demonstration 

PLACEMENT FOLLOW-UP METHODOLOGY- 
OPERATIONAL HIGHLIGHTS 



The test office is notified of its selection for the demonstration and, 
during a predemonstration site review, requested to retain (or obtain) all 
relevant ADS , Job Bank, and/or ESARS reports until the demonstration com- 
mences . 

Employers in the appropriate local area are informed in advance of the 
demonstration and its ramifications through JSIP, ESRs, and/or employer 
groups. 

Job Bank daily Applicant Transaction Listings, ADS Weekly Activity Reports, 
and/or existing ESARS px^jram formats and inquiry functions are used to 
select, in reverse chronological order, a historical sample of 400 place- 
ments . 

Form letters are prepared immediately and sent to all employers of these 
placements, requesting data on placement validity, starting wage, and 
initial (1-3 day) job duration and open-ended comments focusing on quality 
of applicant/placement. 

Nonresponding employers are contacted by telephone after seven calendar 
days to obtain or facilitate their responses. 

Five months thereafter, a second set of form letters is prepared and sent 
to all employers who responded to the first form letter, requesting addi- 
tional data on current/final wages and longer-term (4-150 and 150+ days) 
job duration information, in addition, the respondent is requested to 
verify the responses given to the first form letter. 

Nonresponding employers are contacted by telephone after seven days to 
facilitate or obtain their responses. 

The follow-up data are processed and analyzed along with job order specifi- 
cations and related data. 
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EXHIBIT VII (1) 
MICHIGAN PLACEMENT FOLLOW-UP STUDY — PROCESS DIAGRAM 



I. IDENTIFICATION AND 

SELECTION OF PLACEMENTS 
TO BE FOLLOWED UP 



Obtain Placement Sample from Job 
Bank Daily Applicant Transaction 
Listings, ADS Weekly Activity 
Reports, or ESARS Inquiries 



Obtain Additional 
Placement Data 
from ES-514 
Job Orders 



EL DETERMINATION OF INITIAL 
WAGES .AND JOB DURATIONS 



Prepare Receipt 
Control Log 



Prepare and Send 
4- Day Employer 
Mailers 



Update Receipt 
Control Log 



Review Receipt Control Log >^ 
After 7 Days: Response Received? ~ 



No 




Perform Telephone 
Follow- Up for Non- 
responding 
Employers 



Update Receipt 
Control Log 




Update Receipt 
Control Log I 
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Data Recording, 
Processing, and 
Analysis 



EXHIBIT VII (2) 



DETERMINATION OF LONG-TERM 
WAGES AND JOB DURATIONS 



Review Receipt 
Control Log after 
ISO Days 



i 



Prepare and Send 
ISO- Day Employer 
Mailers 



i 



Update Receipt 
Control Log 



T 



f Review Receipt Control Log No 
^After 7 Days: Response ReceivedTy 



Yes 



Perform Telephone 
Follow- Up for Non- 
responding 
Employee 



t 



Receive Responses 



I Update Receipt j 
j Control Log j 



T 



Update-Receipt 
Control Log 



t 



Data Recording, j 
Processing, and j 
Analysis j 
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EXHIBIT VIII (1) 



Employment and Training Administration 
Referrals/Placements Demonstration 



PUCIiMENT FOLLOW-UP METlJ0D0IOb>-UETAILED SPECIFICATIONS 



Function und iiubf unci ion 


Related Doc uinen L s /H ecor d a 


Procedural Details 


I. IDENTIFICATION AND SELECTION OF 
PIACEMKNTS FOR FOI.U0W-UP 






(1) Select and notify test officii 


. ESARS and/or other national data 

. Letters of notification and instruction 


. Select test office using national data sources 
. Notify test office of .selection and ramifications 
. Advise test office cf specific demonstration plans 
. Request test office to retain daily Applicant Trans- 
action Listings and weekly activity reports for one 
mouth prior to start of demonstration, (If Job 
Hank and ADS reports arc not available, request 
office to perform ESARS inquiries.) 
. Inform local area employers of demonstration through 
JSIP, ESR, or other employer groups 


(2) Select sample placements and 
obtain placement data 


. Daily Job bank Applicant Transaction Listings 
(primary source) 

. ADS weekly activity reports (back-up source) 

. ESARS prog raw formats and inquiry functions (back- 
up source) 

. Data shouts 


. Select placements in reverse chronological order 

from Job Bank, ADS, or ESARS listings 
. Select initial sample of 400 
. Exclude one- to three-day and/or agricultural 

placements (optional) 
. Record applicant name, SSN, placement date, job 

order number, and station or desk identification 

number for responsible interviewer 
, Prepare data sheets 

. Above procedures are performed by project team 


(3) Obtain job order for sample 
placements 


. ES-514 Job Order 
. Data sheuLs 


» P»ecord iob order dpspriotinn uaha dnraHnn [yyp 

code, and number of openings 
. Record employer name, address, telephone, contact, 

and SIC code 
. Record onto same data sheets used in previous 

subf unci ion 

■ Above procedures are performed by project team 


l. DETERMINATION OF INITIAL WAGES AND 
JOB DURATIONS 






(1) Prepare tour-day employer 
mailers 


. Data sheets 

. Standardized mailers 

. Standardized response sheets 


. Obtain following information from data sheets: 
employer name, address, telephone and contact, 
applicant name and SSN, job order number and 
description, and placement date 

♦ Prepare mailer for each employer on agency letter- 
head over office manager's signature 
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EXHIBIT VIII(2) 



Function and Subfuuction 



2. (1) Prepare four-day employer 
maileis (continued) 



U) bund tour-day c-u.-ployer mailers 



I 

to 

00 



by telephone 



M) Receive and proceus responses 



Related Documents/Records 


Procedural Details 




rnfM/iCii v*uQAAn£n uncial at\^o 

• bllUlUbc response envelopes 




• enclose response sneets asKiny ror source 01 




reieiraii jou title, starting date, termination 




uaie f nature or terminal ion , uc drcin9 waye> 




present/final wage, and comments 




. Above procedures are performed by project t/am^ 

/ 




{ 

. Send each employer one mailer along with one 




response sheet for each sample placement 


ft ap Ail if" /'nnfrAl Iao 
• flcLCl^t LUIltlUl luy 


. Send mailers Immediately or tlirce days after most 




reient placement wnicnever is J diet 




. Record date of mailing and to whom sent in control 




1 fin f A r tin pli 1 ah ♦* a V 

1 ocj lot eaui letter 




• nuuve piuueuuies are periormeo uy project team 


UtiP 1 !* 1 lit 1 /VWifr r*Al 1 1 uf 


. Identify nonresponding employers using receipt 




Lontioi j uy 


fill K/vl iifih/i fi' c t" .it utui-m 4* -j i i/i 1 iil: h viifih I /in ij 
• mtlUUIlttOiy bLdLcIllCllL dllU lUStlUlAlUlli) 


. Obtain employer identification information from 




uata sneets 




• La ii any employers troin wnom responses are not 




received after seven days 




. Solicit mail or telephone responses and resolve 




questions or problems 




f it ti r'U 1 ii \ til V n mill li 1 i> A n/i fr* nn />hnv aihaI j k n i. «* . *u 

. line [viewers may also perform outer employer ser - 




\f \ f'AC if o Mini ¥ \ tn& 

viuco at same time 




. Record teleplione contacts in receipt control loy 




4 leiLorm a maximum ot two ceiepnone contact attempts 




tor eacli employer 




• noove procedures are penormeQ Liy to interviewers 




uesiynateu uy test orrice nianayei 


. Response sheets 


, Subfuuction is performed on continuous basis as 


. Data sheets 


responses are received 


. Receipt control log 


. Responses are received in local office by project 




team 




. Project team transcribes answers from response 




sheets to data sheets 




. Project team records responses in receipt control 




log 




. file data sheets and response sheets for subse- 




quent processing 



o 
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EXHIBIT VIII (3) 



Function and Subf unction 


Related Docuttunt s/Records 


Procedural Details 


3, DETERMINATION OF LONG-TERM WAGES 
AND JOB DURATIONS 






(1) Prepare ISC-day employer 
mailer 


. Receipt control log 

. Data sheets 

. Standardised mailers 

. Standardized response sheets 


. Identify employers responding to four-day employer 
mailers using receipt control log 

• uuiui.ii iluui uqLu micuLo a awe jucilL i L y lily llirOiliia 

lion as on four-day employer mailer 

. Prepare mailer for eacli employer responding to four- 
day employer mailers on agency letterhead over 
office manager's signature 

. Enclose response envelopes 

. Enclose response sheets asking for same data as on 
four-day response sheet 

• ruJuvL ^Lutcuuiua die peiiULiucu oy project Lectin 


(2) Send 150-day employer mail una 


• Standardized laaileis 

. Standardized response sheets 

. Rec^iDt rnnlTnl lno 


. Send eacli employer one mailer along with one 

response sheet per placement 
. Send mailers 150 days after most recent placement 

In sample 

. Record date of mailing and to whom sent in control 

log for each letter 
. Above procedures are performed by project team 


(J) Contact non responding employers 
by telephone 


. Receipt control log 
• Data sheets 

. Introductory statement and instructions 


See Subf unction 2(3) <•* «.♦• - 


(4) Receive and process responses 


. Response sheets 

. Data sheets 

• Receipt control log 


See Subfunction 2(4) 
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EVALUATION CRITERIA 



The three types of evaluations proposed for the placement follow-up meth- 
dology are: 

Organizational Evaluation — The objective of this evaluation is to 
assess the impact of the demonstration on local office operations, 
staff, and employers. 

Programmatic Evaluation —Measurements of productivity and per- 
formance from ESARS reports will be used to determine the office 
success in referring applicants, placing referrals, and in filling 
job orders both prior to and during the demonstration period.-. 

Cost-Effectiveness Evaluation — Measurements of cost-effectiveness 
and cost-effectiveness ratios will be computed during the demon- 
stration period. These include: 

Total attempted placement verifications (initial sample) 

~ ~ ~-^tsi*-st^e^f^ 

Short-term response rate (for 4-day follow-up) 

Long-term response rate (for 150-day follow-up) 

Combined response rate 

Staff hours per attempted verification 

Staff 'Hours ' peV'successfuf verification 

Total dollar cost per attempted verification 

Total dollar cost per successful verification 

* * * * 
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IV. ANALYSIS PLAN 



The referral and placement follow-up demonstrations involve both evalua- 
tion and analysis activities. The referral analysis is primarily concerned with 
the dynamics of the referral process, whereas the placement analysis focuses on 
wage and job duration. Several significant benefits can be derived from the 
analyses of both- demonstrations : .... . . 



Quantifies both referral process dynamics and post-placement 
experience on a sample* basis 

JPermits^the isten>£ eta tfo& 

order specifications, local office characteristics and" operations, " 
and related items 

• Reveals the value of the follow-up methodologies*' aT' research ' antf 
analysis tools 

Provides baseline data for future testing and evaluation 

Provides data for making recommendations on referral and place- 
ment follow-up policy 
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